Golf ball tracking system and methods

ABSTRACT

A golf ball tracking system identifies a first pixel at which a golf ball is located in a first image frame and a second pixel at which the golf ball is located in a second image frame. The system may determine that the first pixel represents a location where the golf ball impacts the golf surface based at least in part on a difference in location between the first pixel in the first image frame and the second pixel in the second image frame.

RELATED APPLICATIONS

This application claims priority to U.S. provisional application62/993,344 filed on Mar. 23, 2020, which application is incorporated byreference in its entirety.

BACKGROUND

The present disclosure relates generally to golf ball sensing systems,and more particularly to systems for sensing, calculating and/orotherwise determining a location and/or distance of a golf ball, orother sports object, with respect to one or more physical referencepoints and related methods. Golf is a sport that is continuing to growin popularity. One of golf's main attractions to enthusiasts is thecontinual challenge to improving one's game. To become an adept golferand to maintain golfing proficiency, a significant amount of practice isrequired. However, in order to reap maximum benefit from such practice,it is desirable that the golfer receive feedback on his or herperformance in relative temporal proximity to the performance. Oneexample of such feedback is the location of a golf ball, distancetraveled by the golf ball, and/or the distance from the golf ball to aknown reference point, such as the tee or hole of one or more golfcourse holes and/or driving ranges. Accordingly, there is a need for agolf ball tracking system and related methods.

It should be noted that this Background is not intended to be an aid indetermining the scope of the claimed subject matter nor be viewed aslimiting the claimed subject matter to implementations that solve any orall of the disadvantages or problems presented above. The discussion ofany technology, documents, or references in this Background sectionshould not be interpreted as an admission that the material described isprior art to any of the subject matter claimed herein.

SUMMARY

It is understood that various configurations of the subject technologywill become apparent to those skilled in the art from the disclosure,wherein various configurations of the subject technology are shown anddescribed by way of illustration. As will be realized, the subjecttechnology is capable of other and different configurations and itsseveral details are capable of modification in various other respects,all without departing from the scope of the subject technology.Accordingly, the summary, drawings and detailed description are to beregarded as illustrative in nature and not as restrictive.

In one implementation, a golf ball tracking system comprises a cameraconfigured to capture a plurality of image frames of a golf surface, amemory configured to store the plurality of image frames, and aprocessor. The processor may be configured to identify at least a firstpixel at which a golf ball is located in a first image frame of theplurality of image frames, identify at least a second pixel at which thegolf ball is located in a second image frame of the plurality of imageframes, determine that the first pixel represents a location where thegolf ball impacts the golf surface based at least in part on adifference in location between the first pixel in the first image frameand the second pixel in the second image frame, identify at least athird pixel corresponding to at least a first calibration marker in thefirst image frame, and determine at least one of a location and adistance of the golf ball with respect to a reference point based atleast in part on the location of the first pixel and the location of theat least third pixel in the first image frame. One or more of thecamera, the memory, and the processor may be mounted to and/or disposedwithin an unmanned aerial vehicle.

In another implementation, a golf ball driving range comprises aplurality of targets and at least one unmanned aerial vehicle. Theunmanned aerial vehicle comprises a camera configured to capture aplurality of image frames and a memory configured to store the pluralityof image frames. The golf ball driving range may also comprise aprocessor configured to determine a location of a golf ball on thedriving range with reference to at least one of the plurality oftargets. The processor may be disposed within the unmanned aerialvehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments are discussed in detail in conjunction with theFigures described below, with an emphasis on highlighting theadvantageous features. These embodiments are for illustrative purposesonly and any scale that may be illustrated therein does not limit thescope of the technology disclosed. These drawings include the followingfigures, in which like numerals indicate like parts.

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application with color drawing(s)will be provided by the Office upon request and payment of the necessaryfee.

FIG. 1 illustrates an example of a system for determining, identifyingand/or sensing a location and/or distance of a golf ball with respect toone or more reference points, according to some embodiments;

FIG. 2 illustrates an example block diagram of several examplecomponents of the system of FIG. 1, according to some embodiments;

FIG. 3 illustrates an example kit storing and/or transporting the systemof FIG. 1, according to some embodiments;

FIGS. 4A-4C illustrate portions of successive image frames of a golfingsurface that each include a golf ball, as captured by the system of FIG.1, according to some embodiments;

FIG. 5A illustrates a subtraction mask resulting from subtracting thepixels of the image of FIG. 4A from the image of FIG. 4B, according tosome embodiments;

FIG. 5B illustrates a subtraction mask resulting from subtracting thepixels of the image of FIG. 4B from the image of FIG. 4C, according tosome embodiments;

FIG. 6A illustrates a multiplication mask resulting from multiplying thecolor channels for each pixel of the subtraction mask of FIG. 5A,according to some embodiments;

FIG. 6B illustrates a multiplication mask resulting from multiplying thecolor channels for each pixel of the subtraction mask of FIG. 5B,according to some embodiments;

FIG. 7 illustrates a binary mask of the determined position of the golfball based at least in part on the multiplication masks of FIG. 6A and6B, according to some embodiments;

FIG. 8 illustrates a portion of an image frame of a golfing surfaceincluding a plurality of markers and a golf hole, according to someembodiments;

FIG. 9A illustrates a 2-dimensional plot of the pixel positions of atleast some of the plurality of markers and the golf hole in the imageframe of FIG. 8, according to some embodiments;

FIG. 9B illustrates a 3-dimensional plot of the determined positions ofthe plurality of markers relative to the golf hole in the image frame ofFIG. 8, according to some embodiments;

FIG. 10 illustrates a scatter plot of a determined position of one ofthe plurality markers relative to at least some others of the pluralityof markers in the image frame of FIG. 8, according to some embodiments;

FIG. 11 illustrates at least a portion of an image frame for multi-shottracking at a driving range, according to some embodiments;

FIG. 12 illustrates at least a portion of an image frame for shottracking of multiple players across a portion of or all of a round ofgolf play, according to some embodiments;

FIG. 13 illustrates a flowchart related to a method for tracking a golfball with respect to a reference point, according to some embodiments;

FIG. 14 illustrates an example of a conventional arrangement for sensinga position of a golf ball; and

FIG. 15 illustrates an aerial view of a driving range utilizing aplurality of unmanned aerial vehicles (UAVs) in a system fordetermining, identifying and/or sensing a location and/or distance of agolf ball with respect to one or more reference points, according tosome embodiments.

DETAILED DESCRIPTION

The following detailed description is directed to certain specificembodiments. However, the disclosure can be embodied in a multitude ofdifferent ways. Reference in this specification to “one embodiment” or“an embodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment. The appearances of the phrase “in oneembodiment,” “according to one embodiment,” or “in some embodiments” invarious places in the specification are not necessarily all referring tothe same embodiment, nor are separate or alternative embodimentsmutually exclusive of other embodiments. Moreover, various features aredescribed which may be exhibited by some embodiments and not by others.Similarly, one or more features may be described for one embodimentwhich can also be reasonably used in another embodiment.

FIG. 1 illustrates an example of a system 100 for determining a locationand/or distance of a golf ball with respect to one or more referencepoints, according to some embodiments. In some embodiments, system 100can include a UAV 110 configured to fly over and/or along a golf surface(e.g., a driving range and/or a golf course) and capture a plurality ofimage frames of portions of the golf surface that are ultimatelyutilized to determine a location and/or distance of a golf ball withrespect to one or more reference points (e.g., with respect to a golftee, a golf hole, and/or one or more stationary markers disposed at oneor more locations on, over or along the golf surface). In someembodiments, UAV 110 can be configured with GPS-intelligent flight modesthat allow manual, autonomous, or semi-autonomous flight of UAV 110 toor from a desired location and/or along a desired flight path over oralong the golf surface. An example of such a UAV includes but is notlimited to the DJI Mavic 2 Zoom UAV.

In some embodiments, UAV 110 can include a camera 116 (see FIG. 2)configured to capture image frames at a resolution of, for example, 1080p (i.e., 1920×1080 pixels) and at a frame rate of, for example, 60frames per second (FPS). However, the present disclosure is not solimited and camera 116 of UAV 110 can be configured to capture images ofany suitable or desired resolution and at any suitable or desired framerate. In some cases, camera 116 of UAV 110 can have a 3-axis cameragimbal for stabilizing image capture, though the present disclosure isnot so limited.

Camera 116 of UAV 110 (see FIG. 2) has a field of view (FOV) 102, whichindicates the angle through which camera 116 is configured to captureimage frames of the golf surface below. With increasing cameraresolution comes the ability for UAV 110 to fly at higher altitude andencompass a larger FOV, while still providing sufficient resolution todiscern golf ball positions on the golf surface. For example and notlimitation, if camera 116 of UAV 110 captures images at 1080 presolution (e.g., progressive scan of 1920×1080 pixels per image frame)at an altitude of 125 feet and with an approximate 50 yard×30 yard FOV,a golf ball would be expected to show up as a single pixel in thecaptured image (e.g., roughly assuming 30 yards is 100 feet, a golf ballis approximately 0.1 feet in diameter, and the 30 yard FOV is capturedwith 1080 pixels). Of course, where golf ball detection is carried outwhile the golf ball is in motion, the golf ball may blur across multiplepixels and so the golf ball may affect multiple pixels.

System 100 can further include a camera controller 120 configured tocontrol camera 116 of UAV 110. Controller 120 can be configured towirelessly communicate with at least UAV 110 and, in some cases, displayone or more image frames and/or videos that the camera has captured. Insome embodiments, controller 120 can provide a live view of imagecapture that is controlled by adjusting the head orientation of the userof controller 120. However, the present disclosure is not so limited andcontroller 120 can be configured to control camera 116 of UAV 110according to any suitable method including but not limited to manualcontrols. An example of such a camera controller can include but is notlimited to DJI Racing Goggles.

System 100 can further include a UAV controller 130. UAV controller 130is configured to communicate with UAV 110 and to control flight andoperation of UAV 110, either manually or by inputting one or morecommands that instruct UAV 110 to autonomously or semi-autonomouslyexecute a particular flight plan over and/or along the golf surface.

In some embodiments, UAV controller 130 is further configured to receiveimage data from UAV 110. In some embodiments, UAV 110 can be configuredto store image data in its local memory storage 114 (see FIG. 2) forsubsequent download and/or processing by UAV controller 130 or anothercomputing device. This functionality may relate to a post-processingembodiment, since the image data captured by the camera is stored inmemory 114 of UAV 110 and processed after landing and subsequentdownload of the image data to another computing device (e.g., UAVcontroller 130 and/or computing device 150). Such post-processingembodiments may be advantageous at least in the sense that an algorithmcapable of and configured to process captured image frames in real-timeis not required, since image data may not be processed until UAV 110lands and the data stored thereon is downloaded or otherwise transferredto, e.g., computing device 150. In addition, since the image data neednot be streamed in real-time or near real-time nor processed onboard UAV110, higher resolution video and/or image data can be captured andstored locally in, e.g., memory 114, allowing for potentially moreaccurate golf ball distance and/or location determinations.

In some embodiments, UAV 110 can be configured to stream image data inreal-time or near real-time to UAV controller 130 or another computingdevice (e.g., computing device 150) for processing outside of UAV 110.This functionality may be considered a streaming embodiment, since theimage data captured by camera 116 is either not, or minimally, processedby UAV 110 and is instead wirelessly streamed to UAV controller 130and/or another computing device (e.g., computing device 150) where theimage data is processed. Such streaming embodiments may be advantageousat least in the sense that near-instant golf ball detection can beprovided within the FOV of UAV 110 and the streaming can be integratedwith broadcasting capabilities of wireless communication networks.However, such wireless streaming links tend to be less reliable and moreerror-prone than some other closer-range and/or wired communicationprotocols that may be utilized with the above-described post-processingembodiments. An example of such a UAV controller 130 can include but isnot limited to the DJI Smart Controller, which offers an Occusync™ 2.0protocol that supports high-definition video streaming.

In some embodiments, UAV 110 can be configured to, itself, process theimage data in real-time or near real-time and communicate the calculatedlocation(s) and/or distance(s) of the golf ball(s) to UAV controller 130or another computing device (e.g., computing device 150 and/or userdevice 160). This functionality may be considered an onboard processingembodiment, since the image data captured by camera 116 is processed byone or more processors 112 (see FIG. 2) onboard UAV 110. Such onboardembodiments may be advantageous at least in the sense that near-instantgolf ball detection can be provided and the streaming can be integratedwith broadcasting capabilities or wireless communication networks. Inaddition, since golf ball detection is carried out onboard, fullresolution images need not be streamed, downloaded or otherwisetransmitted via potentially more error-prone, longer-range wirelesscommunication protocols. In some embodiments, such real-time imageprocessing capabilities may involve a heightened minimum processingcapability of the hardware and software onboard UAV 110 compared toeither the post-processing or streaming embodiments described above.

System 100 can further include a computing device 150, which can be apersonal computer, a laptop, a smart device, a server or any othercomputing device suitable for receiving, processing, storing,transmitting and/or displaying image data or any other data related todetermination of a location and/or distance of a golf ball with respectto one or more reference points.

In some embodiments, computing device 150 is connected with UAVcontroller 130 through a streaming capture device 140. In someembodiments, UAV controller 130 communicates with UAV 110 wirelessly andcommunicates with streaming capture device 140 via an HDMI cable, whilestreaming capture device 140 communicates with computing device 150 viaa USB3 cable. However, the present disclosure is not so limited and UAVcontroller 130, streaming capture device 140 and computing device 150can communicate with one another via any suitable medium or mediums,including wireless communication mediums. In some embodiments, UAVcontroller 130 can be configured to communicate image data and/or otherdata directly with computing device 150 without utilizing streamingcapture device 140. In some other embodiments, UAV 110 can be configuredto communicate image and/or other data directly to computing device 150without having to first communicate it through either UAV controller 130or streaming media device 140. In such embodiments, UAV controller 130may function to control the flight and other operations of UAV 110 andnot to mediate communication of image data and/or other data relatedthereto to computing device 150.

System 100 can further include one or more user devices 160, which maybe the user's smartphone but could also or alternatively be any othercomputerized user terminal, configured to receive data regarding thelocation and/or distance of a golf ball with respect to one or morereference points. For example, as will be described in more detail inconnection with one or more of the following figures, user devices 160can include golfers' smartphones and may be configured to run a programor application (i.e., app) that receives at least determined locationand/or distance data of their golf ball from computing device 150.Accordingly, golfers can receive feedback as to their level of play asit relates to the location and/or distance of their golf ball(s) withrespect to one or more reference points (e.g., the golf tee, hole,stationary markers on or along the golf surface, etc.) in real-time ornear real-time on the course or at the driving range.

FIG. 2 illustrates an example block diagram of several examplecomponents of system 100 of FIG. 1, according to some embodiments. UAV110 is illustrated as having one or more processor(s) 112, one or morememories 114, camera 116, a GPS module 115 and a transceiver 118. Theone or more processor(s) 112, alone or in conjunction with memory 114are configured to execute one or more instructions, programs and/orapplications for capturing, saving and/or processing image data fordetermining, identifying and/or sensing a location and/or distance of agolf ball with respect to one or more reference points as describedanywhere in this disclosure. Camera 116 is configured to capture imagedata as described anywhere in this disclosure. GPS module 115 isconfigured to determine the GPS position and/or coordinates of UAV 110,which may be used for navigating and/or otherwise operating UAV 110.Transceiver 118 can comprise at least one antenna, at least onetransmitter and at least one receiver, which are configured, alone or incombination, to communicate image data and/or other data between UAV 110and any one or more of UAV camera controller 120, UAV controller 130,computing device 150, video streamer 140 and user terminal(s) 160 asdescribed anywhere in this disclosure.

UAV camera controller 120 is illustrated as having one or moreprocessor(s) 122, one or more memories 124, one or more input/output(I/O) peripherals 126 and a transceiver 128. The one or moreprocessor(s) 122, alone or in conjunction with memory 124 are configuredto execute one or more instructions, programs and/or applications forcontrolling camera 116 of UAV 110 as described anywhere in thisdisclosure. I/O peripherals 126 can include, but are not limited to, adisplay, a speaker, an accelerometer, a gyroscope or any other controlsfor directing camera 116 of UAV 110, or any other input or output deviceconfigured to receive information and/or other data from or provideinformation and/or other data to a user of UAV camera controller 120.Transceiver 118 can comprise at least one antenna, a transmitter and areceiver, which are configured, alone or in combination, to communicateimage data and/or other control data between UAV camera controller 120and at least UAV 110, as described anywhere in this disclosure.

UAV controller 130 is illustrated as having one or more processor(s)132, one or more memories 134, one or more input/output (I/O)peripherals 136 and a transceiver 138. The one or more processor(s) 132,alone or in conjunction with memory 134 are configured to execute one ormore instructions, programs and/or applications for controlling UAV 110,receiving, processing and/or transmitting image data and/or other data,as described anywhere in this disclosure. I/O peripherals 136 caninclude, but are not limited to, a display, a speaker, an accelerometer,a gyroscope or any other controls for directing UAV 110 and/or fordisplaying image data from camera 116 of UAV 110. Transceiver 118 cancomprise at least one antenna, a transmitter and a receiver, which areconfigured, alone or in combination, to communicate image data and/orother control data between UAV controller 130 and any one or more of UAVcamera controller 120, UAV controller 130, computing device 150, videostreamer 140 and user terminal(s) 160 as described anywhere in thisdisclosure.

Video streamer 140 is illustrated as having one or more processor(s)142, one or more memories 144 and a transceiver 148. The one or moreprocessor(s) 142, alone or in conjunction with memory 144 are configuredto execute one or more instructions, programs and/or applications forultimately streaming image data and/or other data between UAV controller130 and computing device 150 as described anywhere in this disclosure.In some embodiments, transceiver 148 can comprise at least one antenna.In some embodiments, transceiver 148 includes a transmitter and areceiver, configured, alone or in combination, to stream image dataand/or other data between UAV controller 130 and at least computingdevice 150 as described anywhere in this disclosure.

Computing device 150 is illustrated as having one or more processor(s)152, one or more memories 154, one or more input/output (I/O)peripherals 156 and a transceiver 158. The one or more processor(s),alone or in conjunction with memory 154 are configured to execute one ormore instructions, programs and/or applications for determining,identifying and/or sensing a location and/or distance of a golf ballwith respect to one or more reference points as described anywhere inthis disclosure. I/O peripherals 156 can include, but are not limitedto, a display, a keyboard, a speaker, a mouse or any other controls fordetermining, identifying and/or sensing a location and/or distance of agolf ball with respect to one or more reference points as describedanywhere in this disclosure. In some embodiments, transceiver 158 cancomprise at least one antenna, a transmitter and a receiver, which areconfigured, alone or in combination, to receive and/or transmit imagedata, golf ball location and/or distance data, and/or other data fromand/or to any device of system 100 as described anywhere in thisdisclosure.

User terminal(s) 160 is illustrated as having one or more processor(s)162, one or more memories 164, one or more input/output (I/O)peripherals 166 and a transceiver 168. The one or more processor(s) 162,alone or in conjunction with memory 164 are configured to execute one ormore instructions, programs and/or applications for tracking and/ordisplaying determined locations and/or distances of one or more golfballs with respect to one or more reference points as described anywherein this disclosure. I/O peripherals 166 can include, but are not limitedto, a display, a keyboard, a speaker, or any other controls for trackingand/or displaying determined locations and/or distances of one or moregolf balls with respect to one or more reference points as describedanywhere in this disclosure. Transceiver 168 can comprise at least oneantenna, a transmitter and a receiver, which are configured, alone or incombination, to communicate image data and/or other data between userterminal 160 and any device of system 100 as described anywhere in thisdisclosure.

In some embodiments, system 100 can further include one or more markers,as will be described in more detail below, for example, in connectionwith at least FIG. 8.

FIG. 3 illustrates an example kit 300 for storing and/or transportingsystem 100 of FIG. 1, according to some embodiments. In someembodiments, kit 300 may comprise a backpack or other packagingapparatus configured to store UAV 110, UAV camera controller 120, UAVcontroller 130, video streamer 140 and computing device 150. In someembodiments, kit 300 may further store one or more user terminals 160.In some embodiments, kit 300 may further store one or more markers, aswill be described in more detail below in connection with at least FIG.8. Kit 300 may comprise a different storage compartment for each deviceor component of system 100 or, alternatively, may comprise a differentstorage compartment for only a subset of the devices or components ofsystem 100. Kit 300 allows system 100 to be easily transported fromlocation to location, or to be stored between uses.

Image processing carried out by system 100 in order to determinelocations and/or distances of one or more golf balls with respect to oneor more reference points will now be described in connection with FIGS.4A-13.

There are several aspects to determining a location and/or distance of agolf ball with respect to one or more reference points, as contemplatedby the present disclosure. A series of image frames of a golf surfaceare captured by camera 116 of UAV 110. Monitoring and processing of eachcaptured image frame can be carried out by the processor and/or memoryof any one or more of UAV 110, UAV controller 130, computing device 150,or user terminal(s) 160, depending upon whether the above-describedpost-processing, streaming or onboard embodiment(s), or any combinationthereof, is utilized.

Each of the image frames in the series is analyzed for the presence of agolf ball. FIGS. 4A-7 illustrate example embodiments for monitoringimage frames and identifying one or more pixels corresponding to thepresence of a golf ball, according to some embodiments.

Since the orientation of the golf surface and a general direction ofplay is generally known in advance, a predetermined portion of eachimage frame, rather than the entirety of each image frame, can beanalyzed for the presence of a golf ball. In some embodiments, thispredetermined portion can be a top portion of each image frame. However,the present disclosure is not so limited and any predetermined portionof the image frames can be monitored, including the entire image framein some embodiments. This predetermined portion would desirably abut anedge of the image frames, since all golf balls will enter an image framefrom an edge and would be first detected near such an edge. The golfball can be identified as being located at one or more pixels in animage frame.

Any suitable method of image detection can be utilized and iscontemplated by this disclosure. FIGS. 4A-7 illustrate one exampleembodiment that analyzes frame-to-frame motion and color changes toidentify the presence of a golf ball at one or more pixels within animage frame.

FIGS. 4A-4C illustrate portions of three successive image frames 402,404, 406 of a golf surface including a golf ball 410. The pixels of thefirst image frame 402 are subtracted from the corresponding pixels ofthe second image frame 404 and the pixels of the second image frame 404are subtracted from the corresponding pixels of the third image frame406. FIGS. 5A illustrates a subtraction mask 502 resulting from thesubtraction of first image frame 402 from second image frame 404, whileFIG. 5B illustrates a subtraction mask 504 resulting from thesubtraction of second image frame 404 from third image frame 406.Subtraction masks 502, 502 provide the pixel by pixel changes from firstimage 402 to second image 404 and from second image 404 to third image406, respectively.

Since the appearance, disappearance or movement of a golf ball is notthe only potential change that can occur between successive imageframes, subtraction masks 502, 504 may not accurately and reliablyidentify a pixel change in the subtraction masks 502, 504 as indicativeof the presence of a golf ball. However, each subtraction mask 502, 504has a plurality of color channels (e.g., a channel for red content ofthe pixels, a channel for blue content of the pixels and a channel forgreen content of the pixels).

Since a golf ball is customarily white, and white is the presence ofsubstantially equal and/or maximum amounts of red, blue and green light,multiplying the plurality of color channels of subtraction masks 502,504 can amplify the movement of white objects, such as a golf ball,while simultaneously attenuating the movement of other objects.Accordingly, the corresponding pixel values of each of the colorchannels of subtraction mask 502 are multiplied together to producemultiplication mask 602 of FIG. 6A, while the corresponding pixel valuesof each of the color channels of subtraction mask 504 are multipliedtogether to produce multiplication mask 604 of FIG. 6B. As illustratedin the dotted lined boxes 610 a, 610 b of FIGS. 6A, 6B, respectively,pixels where the golf ball was in a previous image frame and no longeris in a next frame, or where the golf ball was not in a previous imageframe but is in the next frame, between image frames 402, 404, 406,produce very small or very large pixel values, respectively, relative tothe average pixel values in multiplication masks 602, 604. Since thepixels where the golf ball is in a particular image frame but is not inthe next successive image frame translates to a color change fromsubstantially white (e.g., the presence of high levels of red, blue andgreen pixel light) to a random color (or a color generally near that ofthe golf surface), pixel(s) in first image frame 402 can be identifiedas indicating the presence of a golf ball at least partly based oncorresponding pixels in multiplication mask 602 having a minimum,substantially minimum, or near minimum value. Accordingly, binary mask702 of FIG. 7 illustrates values indicating presence of a golf ball atthe pixel(s) corresponding to pixels in multiplication mask 602 having aminimum, substantially minimum, or near minimum value. Of course,maximum values could also be utilized to identify the pixel location, orprevious pixel location, of a golf ball.

The above subtraction, multiplication and minimizing procedure can becarried out for each image frame and, e.g., two successive image frames,to determine golf ball pixels in each image frame. Code for carrying outthe procedures, algorithms and/or functions described in this disclosurecan be written and/or executed in any suitable programming language,e.g., C++ with OpenCV and Cuda extensions.

Once a golf ball is detected in an image frame, a region of interest(ROI) of the image is determined (e.g., a subset of pixels that includesthe pixel(s) at which the golf ball is detected). Once a golf ball hasbeen detected and a ROI is determined, the ROI and, in some embodiments,the previously-described predetermined portion, of each image frame issuccessively analyzed from image frame to image frame to identify one ormore pixel(s) in each image frame at which a golf ball is located. Insome embodiments, when a golf ball, that has been previously detected inan image frame and for which an ROI has been determined, is no longerdetected in an image frame (e.g., the golf ball has left the FOV ofcamera 116), the ROI associated with that golf ball may no longer beanalyzed in one or more successive image frames. In this way, and sincea ROI is determined for each golf ball identified in an image frame,multi-shot tracking of multiple golf balls can be simultaneouslyprovided, while reducing the computational burden of golf balldetection.

Once a golf ball has been detected in a series of image frames, a pixelof impact for the golf ball is determined. A pixel of impact can bedefined as the one or more pixels in an image frame where the golf ballis located at the time the golf ball impacts the golf surface afterbeing hit. This pixel of impact can be determined based on decelerationof the golf ball, as determined by comparison between the pixellocations of the golf ball in two successive image frames. For example,a flying golf ball will be identified at pixels in successive imagesthat are displaced from one another by an amount proportional to thegolf ball's speed through the air during the timeframe between thecapture of the two successive image frames. At impact with the golfsurface, the golf ball decelerates greatly. Accordingly, the pixel(s) atwhich the golf ball is identified in the successive image framescaptured immediately after impact with the golf surface will bedisplaced from one another by an amount that is, abruptly, significantlysmaller than between immediately prior successive image frames. Thus,the pixel of impact can be defined as the golf ball pixel in the firstimage frame of successive image frames in which the displacement of thegolf ball pixel from the first image frame to the next image framesubstantially decreases (e.g., decreases by at least a threshold amount,decreases by a greatest or maximum amount, or decreases to substantiallyzero), compared to the displacement of the golf ball pixel between priorsuccessive image frames.

Once the pixel of impact of the golf ball has been identified in animage frame (e.g., the impact frame), the location and/or distance ofthe golf ball with respect to one or more reference points isdetermined. In some embodiments, this determination involves translatingimage frame pixels to real-world location and/or distance values, e.g.,inverse homography. The present disclosure contemplates several methodsfor accomplishing this translation from pixels to real-world golf balllocations and/or distances.

One example option is to utilize GPS of UAV 110 to establish thealtitude and position from which the image frames were captured bycamera 116. With such an option, placement of physical markers on oralong the golf surface to establish scale, context and reference for thepixels of captured images may not be required. Moreover, GPS signals canbe encrypted for increased security and privacy in communications withUAV 110. However, the accuracy of such GPS positioning can vary and GPScoordinates alone do not generally provide substantially accuratedetection of yaw angle of UAV 110, which affects the angle, relative tothe golf surface, at which the image frames are captured and, so,affects the translation of image pixels to real world locations and/ordistances. Accordingly, with such an example option, supplementary yawangle analysis of UAV 110 and/or camera 116 can be beneficial.

Another example option is to use one or more April tags disposed atknown locations on or along the golf surface to determine a referencepoint and a relative orientation between the April tag and UAV 110. Anapril tag is similar to a larger version of a QR code, having a readableblock-encoded surface that encodes a relatively small amount of data(e.g., single to tens of bits) that can be used to determine not onlythe identity of the April tag, but it's relative orientation withrespect to a camera capturing an image of the block-encoded surface.Since the relative shape and dimension of the markings on theblock-encoded surface are predefined, distortions in this relative shapeand dimension of the markings can be used to determine the relativeangle between the camera and the readable block-encoded surface of thecaptured tag. Accordingly, a single April tag may allow foridentification of a reference point within captured image frames and arelative orientation of the UAV 110 and/or camera 116 from the golfsurface and/or the marker surface for translating pixels to real-lifelocations and/or distances. However, April tags can be relatively bulky,having a block-encoded surface large enough to be resolved at suitabledistances and must be placed on or along the golf surface. In somecases, embodiments utilizing a single April tag may still suffer fromrelatively inaccurate yaw detection for UAV 110 by virtue of the smalldimensions of the April tag and inability to resolve small differencesin relative dimensions of the block-encoded surface necessary to detectsufficiently small differences in the relative angle between theblock-encoded surface and camera 116.

Yet another example option is to use the flag at the pin of a golf holeto determine a reference point and a relative orientation between theflag or pin and UAV 110. With such an option, physical markers may notbe required for establishing the reference point and relativeorientation. However, the presence of non-standardized flags or wind cancompromise the ability to provide accurate translations from pixels tolocations and/or distances, altitude of the UAV 110 and accuratedetection of the yaw angle of UAV 110 during capture.

Yet another example option is to use aeropoints, by Propeller Aero, todetermine a reference point and a relative orientation for UAV 110.Aeropoints are configured to report their positions relative to oneanother to within centimeter accuracy utilizing GPS and are solarpowered. However, they are relatively expensive, relatively large inphysical dimension and data they provide must be post-processed, addingto the computational load of any computing device that ultimatelydetermines the locations and/or distances of the golf ball.

Yet another example option is to use inanimate cones or flat markersdisposed at known, static locations and/or distances with respect to oneanother on or along the golf surface to establish reference points forsuch pixel-to-real-world location and/or distance translations. FIG. 8illustrates a portion of an image frame 800 of a golfing surfaceincluding a plurality of markers 802, 804, 806, 808, 814 and a golf hole812, according to some embodiments. As illustrated, each of markers 802,804, 806 and 808 are placed a first predetermined distance from hole 812(e.g., 10 yards-30 feet) or from another desired feature on the golfsurface and squared such that each marker 802, 804, 806, 808 is also asecond predetermined distance (e.g., 42 feet, 5 inches) from each of itsadjacent markers. This ensures markers 802, 804, 806, 808 form a squareor regular diamond centered about golf hole 812. Markers 802, 804, 806,808 can be placed in their proper positions using any suitable method,for example, placing each marker the first predetermined distance fromgolf hole 812 using a first measuring tape and ensuring each marker isalso placed the second predetermined distance from each of its adjacentmarkers using a second measuring tape. It is further contemplated thatmarkers 802, 804, 806, 808, or a subset thereof, can be placed in anyother pattern that similarly allows the establishing of reference pointsfor such pixel-to-real-world location and/or distance translations

To initially or continually verify the accuracy of real-world locationsand/or distances determined from pixel locations, another marker 814 canbe placed at a known location and/or a known distance from any ofmarkers 802. 804, 806, 808 and/or golf hole 812 and the known distanceor location of additional marker 814 can be compared to a calculateddistance or location for additional marker 814 to ensure the two valuessatisfy an appropriate accuracy threshold.

Pixels corresponding to physical calibrated markers, disposed at known,static locations and/or distances with respect to one another on oralong the golf surface can be identified in the impact frame using anysuitable method, including those described above for identifying golfball pixels or similar methods. Since the markers are disposed at known,static locations and/or distances with respect to one another,identified pixels in the impact frame corresponding to these markers canbe used for at least two purposes: (1) to determine a 3-dimensionallocation and orientation of camera 116 of UAV 110 with respect to themarkers, and (2) to determine a real-world location and/or distance ofthe golf ball with respect to one or more reference points (e.g., thegolf tee, a prior location of the golf ball in play on the golf surface,a golf hole, and/or one or more of the above-described calibratedmarkers). For example, the relative orientation of camera 116 of UAV 110can be determined based on a distortion between the relative locationsof pixels in an image frame that correspond to markers 802, 804, 806,808 and the actual, known relative real-world locations of markers 802,804, 806, 808. And this determined relative orientation and the relativelocations of the pixels in the image frame corresponding to markers 802,804, 806, 808 can be used to determine the real-world location (e.g.,distance) of UAV 110 with respect to markers 802, 804, 806, 808. Withthis information, the golf ball pixel in the image frame can be used todetermine a real-world location and/or distance of the golf ball withrespect to one or more of the markers 802, 804, 806, 808, 814, hole 812,or any other point whose position relative to these markers or hole 812is known or determinable.

FIG. 9A illustrates a 2-dimensional plot 900A of identified pixelpositions of markers 802, 804, 806, 808 and golf hole 812, for example,in image frame 800 of FIG. 8, according to some embodiments. FIG. 9Billustrates a 3-dimensional plot 900B of determined or known real-worldpositions of the plurality of markers relative to the golf hole, forexample, in image frame 800 of FIG. 8, according to some embodiments.

Accuracy

There are two components that ultimately determine accuracy of thedisclosed system(s): (1) accuracy of the identification of the pixel ofimpact of a golf ball within an impact frame, and (2) accuracy of thetranslation from pixel of impact to real-world location and/or distancevalues.

Regarding the accuracy of the identification of the pixel of impact ofthe golf ball within the impact frame, an example 1080-pixel FOV thatcovers 30 yards works out to roughly 36 pixels per yard. Assuming a golfball is traveling 100 miles per hour and camera 116 is capturing 1080 pimage frames at a rate of 60 frames per second, the maximum error indetermining the impact pixel position is 15 pixels, or approximately 0.4yards [100 miles per hour×1760 yards per mile=176,000 yards perhour|176,00 yards per hour/3600 seconds per hour/60 frames persecond=0.8148 yards per frame|0.8148 yards per frame×36 pixels=29 pixelsper frame|Since the minimum velocity may be used for determining thecarry distance before impact, the worst case error is the maximum errordivided by two and this condition occurs when the golf ball bouncesexactly between image frames and the local minimum of velocity iseffectively split between two pixel locations].

Regarding the accuracy of the translation from pixel of impact toreal-world location and/or distance values, as previously described inconnection with FIG. 8, another, fifth marker 814 can be placed withinthe FOV and at a known location and/or distance from one or more ofmarkers 802, 804, 806, 808 or golf hole 812. The real-world locationand/or distance of marker 814 can be repeatedly determined based on theidentified pixel locations of the markers 802, 804, 806, 808, 814,similar to how the real-world location of an identified golf ball pixelwould be determined. Because the real-world location of marker 814 isalready known, it can be compared to the calculated real-world locationsof marker 814 to deduce an error associated with pixel-to-real-worldlocation and/or distance in general.

FIG. 10 illustrates a scatter plot 1000 of a plurality of determinedlocations of marker 814 in image frame 800 of FIG. 8, according to someembodiments. As can be seen, the maximum error in pixel-to-yardsconversion is about 2 inches or 0.05 yards.

Inverse Trajectory Generation

Another implementation in which the golf ball location and/or distancedetermining aspects of the present disclosure can be used is drivingrange multi-shot tracking, which allows players to hit a plurality ofgolf balls at, for example, a driving range and receive statistics oftheir shots, in some cases including inverse trajectory generation fortheir shots, downloaded to their smartphone, e.g., user device 160 (seeFIG. 1). FIG. 11 illustrates at least a portion of an image frame 1100for such multi-shot tracking, according to some embodiments. Asillustrated, multiple impact pixels 1102, 1104, 1106 are identified formultiple golf balls on the same image frame. The dotted lines 1112,1114, 1116 may correspond to inverse trajectories for each golf ball,generated based at least in part on the identified impact pixels 1102,1104, 1106 and/or golf ball pixel identification from and between priorand/or subsequent image frames. In other words, a trajectory of analready-hit golf ball can be calculated after being hit, based on itsdirection, speed, acceleration and/or the impact characteristics of thegolf ball with the golf surface.

Yet another implementation in which the golf ball location and/ordistance determining aspects of the present disclosure can be used is ina golf assistant, which allows players to track their play for an entireround of golf, or a subset thereof, on their smartphone (e.g., userdevice 160, see FIG. 1). In such implementations, a dedicated UAV 110may be dispatched over each or at least multiple golf holes of thecourse and user device 160 for a particular golfer can be configured tocontinuously or intermittently stream or otherwise transmitaccelerometer and/or GPS information in real-time or otherwise to one ormore of UAV 110 and/or computing device 150, depending upon where golfball location determination is performed. As a new golf ball pixel isidentified based on detection of a moving golf ball within an imageframe, the one or more of UAV 110 and/or computing device 150 candetermine which player just hit the golf ball based at least in part onthe accelerometer and/or GPS data from user device 160 corresponding toeach golfer, and/or generation of an inverse trajectory of the golf ballas described above at least in connection with FIG. 11 and, in somecases, further including the known acceleration characteristics and GPScoordinates of the golfers before, during and after a golf ball is hit.In some cases, golf club contact with a golf ball can be analyzed basedon a backswing of the golfer. FIG. 12 illustrates at least a portion ofan image frame 1200 for shot tracking of multiple players 1202, 1204across a portion or all of a round of golf play, according to someembodiments.

Multiple UAV Rotation

The present disclosure also contemplates the simultaneous utilization ofa plurality of such UAVs 110 on a golf course, driving range or anyother playing surface. In some such embodiments, each UAV 110 may beconfigured to fly, hover and/or service one or more targets, e.g., golfholes, distance targets, and fly back to a local charging station whenone or more conditions are met, for example, a battery charge of UAV 110falls below a predetermined and/or calculated threshold or an error isdetected in one or more operations associated with UAV 110. In this way,multiple targets can be serviced simultaneously and such service cancontinue uninterrupted, potentially indefinitely, so long as there isanother UAV 110 available to take over service when a particular UAV istriggered and/or determines to fly back to a charging station and/or toanother service location.

Driving Range Targets

FIG. 14 illustrates an example of a conventional arrangement 1400 forsensing a position of one or more golf balls 1406. Arrangement 1400includes a plurality of targets 1402, for example, golf holesparticipants can aim for when driving golf balls 1406. However, in orderto accurately sense a position of golf ball 1406, arrangement 1400 alsorequires that complex electronic ball-detection systems 1404 beinstalled in the vicinity and/or in relative proximity to each oftargets 1402.

In some such arrangements, electronic ball-detection systems 1404 areconfigured to sense a particular electronic, electromagnetic and/ormagnetic quality associated with golf balls 1406. Accordingly, golfballs 1406 are also required to possess that particular electronic,electromagnetic and/or magnetic quality, which a regular golf ball doesnot normally possess, to enable proximately located electronicball-sensing systems 1404 to accurately detect the position of golfballs 1406. For example, golf balls 1406 may be required to include atleast some magnetic material, electrically conductive material, amaterial substantially opaque to radar or other reflection-basedlocating signal, radio frequency identification (RFID) tags, near-fieldcommunication (NFC) tags or some other feature that allows electronicsystems 1404 to sense the presence and/or proximity of golf balls 1406.

This one-to-one requirement for electronic ball-detection systems 1404to service multiple targets 1402 and/or for specially designed golfballs 1406 can make arrangement 1400 prohibitively expensive toimplement. Moreover, if a particular electronic system 1404 fails orfalls out of calibration, or if a user drives regular golf balls inarrangement 1400 without understanding the above-described requirements,the driving range is effectively rendered inoperable for its intendedpurpose, at least with respect to the failed ball-detection system 1404and/or the regular golf balls.

The present disclosure overcomes these inconveniences and limitations atleast in part by utilizing one or more unmanned aerial vehicles (UAVs)to determine a location and/or distance of any sport object, forexample, a regular golf ball, with respect to one or more referencepoints, as will be described in more detail below.

FIG. 15 illustrates an aerial view of a driving range utilizing aplurality of drones 110 a, 110 b in a system 1500 for determining,identifying and/or sensing a location and/or distance of a golf ballwith respect to one or more reference points, according to someembodiments. While several features of system 1500 are described belowfor ease of understanding, it should be understood that system 1500 mayadditionally include any aspect, feature and/or equipment describedanywhere else in this disclosure.

System 1500 includes a plurality of targets 1502, which may be, forexample, golf holes and/or distance targets of a golf course and/ordriving range. System 1500 further includes a plurality of chargingstations 1510 disposed at varied locations along, on or outside adesignated playing area of such a golf course and/or driving range.System further includes a plurality of UAVs 110 a, 110 b, one or morebeing a currently inactive UAV 110 a and parked and/or otherwisedisposed on a charging station 1510, and one or more being a currentlyactive UAV 110 b flying on one or more suitable flight plans and/orhovering at one or more suitable locations for determining, identifyingand/or sensing a location and/or distance of a golf ball with respect toone or more reference points, as described anywhere in this disclosure.

As illustrated, while active, one or more of UAVs 110 b may service oneor more designated targets 1502 as described anywhere in thisdisclosure. While inactive, one or more UAVs 110 a are being charged ona respective charging station 1510 and/or being serviced in some otherway. When a suitable condition occurs, one or more inactive UAVs 110 amay activate and fly to a suitable location to relieve and/or take overthe function(s) and/or responsibilities of one or more other currentlyactive UAVs 110 b. In some embodiments, a currently active UAV 110 b mayterminate performing its current function(s) and/or responsibilitiesimmediately upon sensing such a suitable condition, or after apredetermined or otherwise calculated delay of sensing such a suitablecondition, with or without notification that another UAV is available totake over. Examples of such a suitable condition include but are notlimited to one or more of inactive UAVs 110 a becoming sufficientlycharged, one or more inactive UAVs 110 a having been serviced and nowbeing operational or capable of proper operation, one or more activeUAVs 110 b sensing a battery charge has fallen below a threshold, one ormore active UAVs 110 b sensing an operating error, or one or more activeor inactive UAVs 110 a, 110 b receiving a command to terminate, initiateor exchange responsibilities based on any factor, including but notlimited to human control, a time of day, a number of golf balls detectedin a session by a particular UAV or by system 1500 as a whole, or a userinitiating, terminating or pausing a particular sporting session.

Methods

FIG. 13 illustrates a flowchart 1300 related to a method fordetermining, identifying and/or sensing a location and/or distance of agolf ball with respect to a reference point, according to someembodiments. Although particular steps, actions or blocks are describedbelow. The present disclosure is not so limited and one or moreadditional or alternative steps, actions or blocks may also be utilized,one or more steps, actions or blocks may be omitted, or a differentordering may be utilized without departing from the scope of thedisclosure.

Flowchart 1300 may correspond to operations of any one or more of UAV110, UAV controller 130, computing device 150, user terminal 160, and/orspecifically to operations of any one or more of the processor(s),memories, input/output peripherals, and/or transceivers of thosedevices, alone or in combination.

Flowchart 1300 includes block 1302, which includes capturing a pluralityof image frames of a golf surface with a camera. For example, aspreviously described, UAV 110 comprises camera 116, which is configuredto capture a plurality of image frames (e.g., 402, 404, 406 of FIGS.4A-4C) of a golf surface.

Flowchart 1300 proceeds to block 1304, which includes storing theplurality of image frames in a memory storage. For example, aspreviously described, image frames can be stored in any of memories 114,124, 134, 144, 154 or 164 of UAV 110, UAV camera controller 120, UAVcontroller 130, video streamer 140, and/or user terminal(s) 160,respectively (see FIGS. 1 and 2).

Flowchart 1300 proceeds to block 1306, which includes identifying,utilizing a processor, at least a first pixel at which a golf ball islocated in a first image frame of the plurality of image frames. Forexample, as previously described, any of processors 112, 132, 152 and162 of UAV 110, UAV controller 130, computing device 150 and userterminal(s) 160, respectively, can identify at least a first pixel(e.g., corresponding to golf ball 410) at which a golf ball is locatedin first image frame 402.

In some embodiments, identifying at least the first pixel at which thegolf ball is located in first image frame 402 comprises obtaining afirst subtraction mask 502 (see FIG. 5A) by subtracting correspondingpixels of first image frame 402 from corresponding pixels of secondimage frame 404. First subtraction mask 502 comprises a plurality ofcolor channels, each color channel comprising pixel values indicating anamount of a respective color contained by each pixel in the firstsubtraction mask. The identifying at least the first pixel can furthercomprise obtaining a first multiplication mask 602 (see FIG. 6A) by, foreach pixel in first subtraction mask 502, multiplying the amounts ofeach respective color indicated by each color channel, and selecting thefirst pixel in first image frame 402 as corresponding to the pixelhaving one of the highest value and the lowest value in firstmultiplication mask 602.

Flowchart 1300 proceeds to block 1308, which includes identifying,utilizing the processor, at least a second pixel at which the golf ballis located in a second image frame of the plurality of image frames. Forexample, as previously described, any of processors 112, 132, 152 and162 of UAV 110, UAV controller 130, computing device 150 and userterminal(s) 160, respectively, can identify at least a second pixel(e.g., corresponding to golf ball 410) at which a golf ball is locatedin second image frame 404.

In some embodiments, identifying at least the second pixel at which thegolf ball is located in second image frame 404 comprises obtaining asecond subtraction mask 504 (see FIG. 5B) by subtracting correspondingpixels of second image frame 404 from corresponding pixels of a thirdimage frame 406. Second subtraction mask 504 comprises a plurality ofcolor channels, each color channel comprising pixel values indicating anamount of a respective color contained by each pixel in the secondsubtraction mask. The identifying at least the second pixel can furthercomprise obtaining a second multiplication mask 604 (see FIG. 6B) by,for each pixel in second subtraction mask 504, multiplying the amountsof each respective color indicated by each color channel, and selectingthe second pixel in second image frame 404 as corresponding to the pixelhaving one of the highest value and the lowest value in secondmultiplication mask 604.

Flowchart 1300 proceeds to block 1310, which includes determining,utilizing the processor, that the first pixel represents a locationwhere the golf ball impacts the golf surface based at least in part on adifference in location between the first pixel in the first image frameand the second pixel in the second image frame. For example, aspreviously described, any of processors 112, 132, 152 and 162 of UAV110, UAV controller 130, computing device 150 and user terminal(s) 160,respectively, can determine that the first pixel represents a locationwhere golf ball 410 impacts the golf surface based at least in part on adifference in location between the first pixel in first image frame 402and the second pixel in second image frame 404.

In some embodiments, the determining step of block 1310 comprisesdetermining that a first difference in location between the first pixelin first image frame 402 and the second pixel in second image frame 404is smaller than a second difference in location between a pixel at whichthe golf ball is located in an image frame of the plurality of imageframes captured immediately prior to the first image frame and the firstpixel in first image frame 402.

Flowchart 1300 proceeds to block 1312, which includes identifying,utilizing the processor, at least a third pixel corresponding to atleast a first calibration marker in the first image frame. For example,as previously described, any of processors 112, 132, 152 and 162 of UAV110, UAV controller 130, computing device 150 and user terminal(s) 160,respectively, can identify at least a third pixel corresponding to atleast a first calibration marker (e.g., any of markers 802, 804, 806,808 of FIG. 8) in first image frame 402.

Flowchart 1300 proceeds to block 1314, which includes determining,utilizing the processor, at least one of a location and a distance ofthe golf ball with respect to a reference point based at least in parton the location of the first pixel and the location of the at leastthird pixel in the first image frame. For example, as previouslydescribed, any of processors 112, 132, 152 and 162 of UAV 110, UAVcontroller 130, computing device 150 and user terminal(s) 160,respectively, can determine at least one of a location and a distance ofgolf ball 410 with respect to a reference point (e.g., any one ofmarkers 802, 804, 806, 808 and golf hole 812) based at least in part onthe location of the first pixel (e.g., corresponding to golf ball 410)and the location of the at least third pixel (e.g., corresponding to anyof markers 802, 804, 806, 808 of FIG. 8) in first image frame 402.

In some embodiments, flowchart 1300 could further include streaming theplurality of image frames to one or more of a controller of the unmannedaerial vehicle and the computing device. For example, in some streamingembodiments as described above in connection with at least FIGS. 1 and2, image data may be streamed from UAV 110 to, e.g., to UAV controller130 and/or computing device 150 while UAV 110 is flying. Accordingly,processing of the image frames can take place by UAV controller 130and/or computing device 150 in real-time or near real-time while UAV 110is in flight, even in circumstances wherein UAV 110 is not configured orcapable of processing the captured image frames in real-time or nearreal-time.

In some embodiments, flowchart 1300 could further include uploading theplurality of image frames to one or more of a controller of the unmannedaerial vehicle and the computing device after the unmanned aerialvehicle lands. For example, in some post-processing embodiments asdescribed above in connection with at least FIGS. 1 and 2, image datamay not be streamed from UAV 110 to another computing device while UAV110 is flying and is, instead, uploaded to UAV controller 130 and/orcomputing device 150 upon UAV landing. Processing of the image framescan take place by UAV controller 130 and/or computing device 150 afterUAV 110 lands, accordingly.

In some embodiments, flowchart 1300 could further include determining aregion of interest, based at least in part on identifying the firstpixel in first image frame 402, in which pixels of subsequent imageframes (e.g., image frames 404, 406) of the plurality of image framesare analyzed for identifying a subsequent location of golf ball 410. Forexample, as previously described, narrowing analysis of image frames toonly pixels in one or more regions of interest, which can include apredetermined portion of the image frames in which a golf ball isexpected or highly likely to enter the image frames can reduce thecomputational load of analyzing image frames for the presence of golfballs.

In some embodiments, flowchart 1300 could further include determining a3-dimensional location of camera 116 based at least in part on adistortion between an orientation of the respective pixels in the firstimage frame corresponding to each of the plurality of calibrationmarkers and a known orientation of the plurality of calibration markers.For example, as previously described in connection with at least FIG. 8,a 3-dimensional location of camera 116 can be determined based at leastin part on a distortion between an orientation of the respective pixelsin first image frame 402 corresponding to each of calibration markers802, 804, 806, 808 and a known orientation of those markers.

In some embodiments, flowchart 1300 could further include verifying anaccuracy of the determined location or distance of the golf ball withrespect to the reference point by determining a location of theauxiliary calibration marker and comparing the determined location ofthe auxiliary calibration marker with the predetermined location. Forexample, as previously described in connection with at least FIG. 8, anaccuracy of the determined location or distance of golf ball 410 withrespect to a reference point (e.g., any of markers 802, 804, 806, 808and/or hole 812) can be verified by determining a location of theauxiliary calibration marker 812, and comparing the determined locationof marker 812 with its predetermined location.

In some embodiments, flowchart 1300 could further include generating aninverse trajectory of the golf ball based at least in part on the firstpixel at which the golf ball is located in the first image frame and thesecond pixel at which the golf ball is located in the second imageframe. For example, as previously described in connection with at leastFIG. 11, an inverse trajectory 1112, 1114, 1116 of any of golf balls1102, 1104, 1106 can be generated based at least in part on the firstpixel at which the golf ball is located in first image frame 402 (seeFIG. 4) and the second pixel at which the golf ball is located in secondimage frame 404 (see FIG. 4).

In some embodiments, flowchart 1300 could further include determiningwhich of a plurality of golfers hit the golf ball based at least in parton the generated inverse trajectory of the golf ball and at least one ofaccelerometer data and global positioning method data corresponding toat least one of the plurality of golfers. For example, as previouslydescribed in connection with at least FIG. 12, a golfer (e.g., one ofgolfers 1202, 1204 of FIG. 12) that hit the golf ball can be identifiedbased at least in part on the generated inverse trajectory of the golfball and at least one of accelerometer data and global positioningmethod data corresponding to at least one of the golfers 1202, 1204.

General Interpretive Principles for the Present Disclosure

Various aspects of the novel systems, apparatuses, and methods aredescribed more fully hereinafter with reference to the accompanyingdrawings. The teachings disclosure may, however, be embodied in manydifferent forms and should not be construed as limited to any specificstructure or function presented throughout this disclosure. Rather,these aspects are provided so that this disclosure will be thorough andcomplete, and will fully convey the scope of the disclosure to thoseskilled in the art. Based on the teachings herein one skilled in the artshould appreciate that the scope of the disclosure is intended to coverany aspect of the novel systems, apparatuses, and methods disclosedherein, whether implemented independently of or combined with any otheraspect of the disclosure. For example, a system or an apparatus may beimplemented, or a method may be practiced using any one or more of theaspects set forth herein. In addition, the scope of the disclosure isintended to cover such a system, apparatus or method which is practicedusing other structure, functionality, or structure and functionality inaddition to or other than the various aspects of the disclosure setforth herein. It should be understood that any aspect disclosed hereinmay be set forth in one or more elements of a claim. Although somebenefits and advantages of the preferred aspects are mentioned, thescope of the disclosure is not intended to be limited to particularbenefits, uses, or objectives. The detailed description and drawings aremerely illustrative of the disclosure rather than limiting, the scope ofthe disclosure being defined by the appended claims and equivalentsthereof.

With respect to the use of plural vs. singular terms herein, thosehaving skill in the art can translate from the plural to the singularand/or from the singular to the plural as is appropriate to the contextand/or application. The various singular/plural permutations may beexpressly set forth herein for sake of clarity.

When describing an absolute value of a characteristic or property of athing or act described herein, the terms “substantial,” “substantially,”“essentially,” “approximately,” and/or other terms or phrases of degreemay be used without the specific recitation of a numerical range. Whenapplied to a characteristic or property of a thing or act describedherein, these terms refer to a range of the characteristic or propertythat is consistent with providing a desired function associated withthat characteristic or property.

In those cases where a single numerical value is given for acharacteristic or property, it is intended to be interpreted as at leastcovering deviations of that value within one significant digit of thenumerical value given.

If a numerical value or range of numerical values is provided to definea characteristic or property of a thing or act described herein, whetheror not the value or range is qualified with a term of degree, a specificmethod of measuring the characteristic or property may be defined hereinas well. In the event no specific method of measuring the characteristicor property is defined herein, and there are different generallyaccepted methods of measurement for the characteristic or property, thenthe measurement method should be interpreted as the method ofmeasurement that would most likely be adopted by one of ordinary skillin the art given the description and context of the characteristic orproperty. In the further event there is more than one method ofmeasurement that is equally likely to be adopted by one of ordinaryskill in the art to measure the characteristic or property, the value orrange of values should be interpreted as being met regardless of whichmethod of measurement is chosen.

It will be understood by those within the art that terms used herein,and especially in the appended claims (e.g., bodies of the appendedclaims) are intended as “open” terms unless specifically indicatedotherwise (e.g., the term “including” should be interpreted as“including but not limited to,” the term “having” should be interpretedas “having at least,” the term “includes” should be interpreted as“includes but is not limited to,” etc.).

It will be further understood by those within the art that if a specificnumber of an introduced claim recitation is intended, such an intentwill be explicitly recited in the claim, and in the absence of suchrecitation no such intent is present. For example, as an aid tounderstanding, the following appended claims may contain usage of theintroductory phrases “at least one” and “one or more” to introduce claimrecitations. However, the use of such phrases should not be construed toimply that the introduction of a claim recitation by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim recitation to embodiments containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should typically be interpreted to mean “atleast one” or “one or more”); the same holds true for the use ofdefinite articles used to introduce claim recitations. In addition, evenif a specific number of an introduced claim recitation is explicitlyrecited, those skilled in the art will recognize that such recitationshould typically be interpreted to mean at least the recited number(e.g., the bare recitation of “two recitations,” without othermodifiers, typically means at least two recitations, or two or morerecitations).

In those instances where a convention analogous to “at least one of A,B, and C” is used, such a construction would include systems that have Aalone, B alone, C alone, A and B together without C, A and C togetherwithout B, B and C together without A, as well as A, B, and C together.It will be further understood by those within the art that virtually anydisjunctive word and/or phrase presenting two or more alternative terms,whether in the description, claims, or drawings, should be understood tocontemplate the possibilities of including one of the terms, either ofthe terms, or both terms. For example, the phrase “A or B” will beunderstood to include A without B, B without A, as well as A and Btogether.”

Various modifications to the implementations described in thisdisclosure can be readily apparent to those skilled in the art, andgeneric principles defined herein can be applied to otherimplementations without departing from the spirit or scope of thisdisclosure. Thus, the disclosure is not intended to be limited to theimplementations shown herein but is to be accorded the widest scopeconsistent with the claims, the principles and the novel featuresdisclosed herein. The word “exemplary” is used exclusively herein tomean “serving as an example, instance, or illustration.” Anyimplementation described herein as “exemplary” is not necessarily to beconstrued as preferred or advantageous over other implementations.

Certain features that are described in this specification in the contextof separate implementations also can be implemented in combination in asingle implementation. Conversely, various features that are describedin the context of a single implementation also can be implemented inmultiple implementations separately or in any suitable sub-combination.Moreover, although features can be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination can be directed to asub-combination or variation of a sub-combination.

The methods disclosed herein comprise one or more steps or actions forachieving the described method. The method steps and/or actions may beinterchanged with one another without departing from the scope of theclaims. In other words, unless a specific order of steps or actions isspecified, the order and/or use of specific steps and/or actions may bemodified without departing from the scope of the claims.

What is claimed is:
 1. A golf ball tracking system, comprising: a cameraconfigured to capture a plurality of image frames of a golf surface; amemory configured to store the plurality of image frames; a processorconfigured to: identify at least a first pixel at which a golf ball islocated in a first image frame of the plurality of image frames;identify at least a second pixel at which the golf ball is located in asecond image frame of the plurality of image frames; determine that thefirst pixel represents a location where the golf ball impacts the golfsurface based at least in part on a difference in location between thefirst pixel in the first image frame and the second pixel in the secondimage frame; identify at least a third pixel corresponding to at least afirst calibration marker in the first image frame; and determine atleast one of a location and a distance of the golf ball with respect toa reference point based at least in part on the location of the firstpixel and the location of the at least third pixel in the first imageframe.
 2. The system of claim 1, wherein the camera is mounted to anunmanned aerial vehicle.
 3. The system of claim 1, wherein the memory isdisposed within an unmanned aerial vehicle.
 4. The system of claim 3,wherein the processor is disposed within the unmanned aerial vehicle. 5.The system of claim 3, wherein the processor is disposed within acomputing device other than the unmanned aerial vehicle.
 6. The systemof claim 5, wherein the unmanned aerial vehicle further comprises atransceiver configured to stream the plurality of image frames to one ormore of a controller of the unmanned aerial vehicle and the computingdevice.
 7. The system of claim 5, wherein the unmanned aerial vehicle isconfigured to upload the plurality of image frames to one or more of acontroller of the unmanned aerial vehicle and the computing device afterthe unmanned aerial vehicle lands.
 8. The system of claim 1, wherein theprocessor is configured to identify at least the first pixel at whichthe golf ball is located in the first image frame of the plurality ofimage frames by: obtaining a first subtraction mask by subtractingcorresponding pixels of the first image frame from corresponding pixelsof the second image frame, the first subtraction mask comprising aplurality of first color channels, each first color channel comprisingpixel values indicating an amount of a respective color contained byeach pixel in the first subtraction mask; obtaining a firstmultiplication mask by, for each pixel in the first subtraction mask,multiplying the amounts of each respective color indicated by each firstcolor channel; and selecting the first pixel in the first image frame ascorresponding to the pixel having one of the highest value and thelowest value in the first multiplication mask.
 9. The system of claim 1,wherein the processor is configured to identify at least the secondpixel at which the golf ball is located in the second image frame of theplurality of image frames by: obtaining a second subtraction mask bysubtracting corresponding pixels of the second image frame fromcorresponding pixels of a third image frame of the plurality of imageframes, the second subtraction mask comprising a plurality of secondcolor channels, each second color channel comprising pixel valuesindicating an amount of a respective color contained by each pixel inthe second subtraction mask; obtaining a second multiplication mask by,for each pixel in the second subtraction mask, multiplying the amountsof each respective color indicated by each second color channel; andselecting the second pixel in the second image frame as corresponding tothe pixel having one of the highest value and the lowest value in thesecond multiplication mask.
 10. The system of claim 1, wherein theprocessor is configured to determine that the first pixel represents thelocation where the golf ball impacts the golf surface by: determiningthat a first difference in location between the first pixel in the firstimage frame and the second pixel in the second image frame is smallerthan a second difference in location between a pixel at which the golfball is located in an image frame of the plurality of image framescaptured immediately prior to the first image frame and the first pixelin the first image frame.
 11. The system of claim 1, wherein theprocessor is configured to determine a region of interest, based atleast in part on identifying the first pixel in the first image frame,in which pixels of subsequent image frames of the plurality of imageframes are analyzed for identifying a subsequent location of the golfball.
 12. The system of claim 11, wherein the processor identifies thesecond pixel in the second image frame based on analyzing the pixelswithin the region of interest in the second image frame.
 13. The systemof claim 1, wherein the at least a first calibration marker comprises aplurality of calibration markers and the processor is configured todetermine respective pixels in the first image frame corresponding toeach of the plurality of calibration markers, each calibration markerdisposed a first predetermined distance from the reference point and asecond predetermined distance from at least one other of the pluralityof calibration markers.
 14. The system of claim 13, wherein thereference point is a golf hole.
 15. The system of claim 13, wherein theprocessor is configured to determine a 3-dimensional location of thecamera based at least in part on a distortion between an orientation ofthe respective pixels in the first image frame corresponding to each ofthe plurality of calibration markers and a known orientation of theplurality of calibration markers.
 16. The system of claim 15, whereinthe processor is configured to determine the at least one of thelocation and the distance of the golf ball with respect to the referencepoint based at least in part on the location of the first pixel and the3-dimensional location of the camera.
 17. The system of claim 13,wherein the plurality of calibrated markers comprises an auxiliarycalibration marker disposed at a predetermined location with respect toat least one of the other plurality of calibration markers, theprocessor configured to verify an accuracy of the determined location ordistance of the golf ball with respect to the reference point by:determining a location of the auxiliary calibration marker; andcomparing the determined location of the auxiliary calibration markerwith the predetermined location.
 18. The system of claim 1, wherein theprocessor is configured to generate an inverse trajectory of the golfball based at least in part on the first pixel at which the golf ball islocated in the first image frame and the second pixel at which the golfball is located in the second image frame.
 19. The system of claim 18,wherein the processor is configured to determine which of a plurality ofgolfers hit the golf ball based at least in part on the generatedinverse trajectory of the golf ball and at least one of accelerometerdata and global positioning system data corresponding to at least one ofthe plurality of golfers.
 20. A golf ball driving range, comprising: aplurality of targets; at least one unmanned aerial vehicle, wherein theunmanned aerial vehicle comprises: a camera configured to capture aplurality of image frames; and a memory configured to store theplurality of image frames; and; a processor configured to determine alocation of a golf ball on the driving range with reference to at leastone of the plurality of targets.
 21. The method of claim 20, wherein theprocessor is disposed within the unmanned aerial vehicle.